Image processing method and display device

ABSTRACT

An image processing method to obtain a high sense of depth or high stereoscopic effect for an image and a display device utilizing the method are provided. Image data of an image is separated into image data of a plurality of objects and a background. A feature amount is obtained from the image data of each object, so that the objects are identified. The relative distance between viewer&#39;s eye and any of the objects is determined by the data of the sizes of the objects in the image and the sizes of the objects stored in the database. The image data of each object is processed so that an object with a shorter relative distance is enlarged. The image data of each object after image processing is combined with the image data of the background, so that a sense of depth or stereoscopic effect of an image is increased.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/091,622, filed Apr. 6, 2016, now allowed, which is a continuation ofU.S. application Ser. No. 13/427,086, filed Mar. 22, 2012, now U.S. Pat.No. 9,313,479, which claims the benefit of a foreign priorityapplication filed in Japan as Serial No. 2011-067069 on Mar. 25, 2011,all of which are incorporated by reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

One embodiment of the present invention relates to an image processingmethod and a display device utilizing the image processing method.

2. Description of the Related Art

The market for three-dimensional display devices is growing. Displayinga three-dimensional image can be achieved by artificially creating, witha display device, difference between retinal images of both eyes(binocular parallax) which may occur when a viewer sees a stereoscopicobject with both eyes. The three-dimensional image display devicesutilizing the binocular parallax have been developed and commercializedwith a variety of display methods. The variety of display methods ismainly classified into a direct-view display method utilizing an opticalsystem such as a parallax barrier, a lenticular lens, or a microlensarray and a display method utilizing glasses with shutters.

Patent Document 1 discloses a technique for displaying athree-dimensional image by a parallax barrier so that a right eye seesan image for the right eye and a left eye sees an image for the lefteye. Patent Document 2 discloses a liquid crystal display device thatdisplays three-dimensional images utilizing glasses.

REFERENCE Patent Document

-   -   [Patent Document 1] Japanese Published Patent Application No.        H8-036145    -   [Patent Document 2] Japanese Published Patent Application No.        2003-259395

SUMMARY OF THE INVENTION

In a three-dimensional image display method utilizing an optical systemsuch as a parallax barrier, a lenticular lens, or a microlens array,light from a pixel that corresponds to a right eye enters a right eyeand light from a pixel that corresponds to a left eye enters a left eye.Consequently, the number of pixels that contribute to image display, inthe horizontal direction of a pixel portion, is reduced to half of theactual number, which prevents high-definition images from beingdisplayed.

In a three-dimensional image display method utilizing glasses, an imagefor a right eye and an image for a left eye are alternately displayed ona screen. By seeing the images through glasses with shutters, human eyesrecognize that the images are three-dimensional. In thethree-dimensional image display device, as compared to thetwo-dimensional image display device, the number of writing operationsof images to a pixel portion during one frame period is increased;accordingly, a high-performance driver circuit that is capable ofdriving at high frequency is needed and power consumption of the wholedisplay device is increased.

In view of the above problems, one embodiment of the present inventionprovides an image processing method and a display device utilizing theimage processing method. In the image processing method, a sense ofdepth or stereoscopic effect of images is increased while the number ofwriting of images to a pixel portion during one frame period is reducedor the number of pixels that contribute to image display is not reduced.

One embodiment of the present invention utilizes human perception ofperspective: in the case where a plurality of objects exists in atwo-dimensional image and an object is enlarged more than the otherobjects, the object seems to be located more on the front side than theother objects. In the description, a front side of an image is a part ofthe image that a viewer will perceive as closer from him than a backside of the image. Further, in the description, distances may have to beunderstood as “distances perceived by a viewer” or “intended perceiveddistances by a user” according to the context.

Specifically, in an image processing method according to one embodimentof the present invention, image data is separated into image data of aplurality of objects and image data of a background. A feature amount isobtained from the image data of each object. A feature amount isobtained by converting features, which are useful for object recognitionof the object into values, and objects can be identified by the featureamount. For example, a feature amount obtained from a shape, a color, agray scale, texture, or the like of an object can be used foridentification of the object. Then, each of the feature amounts iscompared with data, stored in database, of the size of an object that isa model correlated with the feature amount of the object that is amodel, so that data of the size of each object is obtained. A relativedistance between viewer's eye and any of the objects or a front-backrelation among the objects can be determined by the data of the size ofeach object obtained from the database and the relative sizes of theobjects in the image. Then, the image data of each object is processedso that an object that has a shorter relative distance from viewer's eyeor an object that is located more on the front side is enlarged. Then,the image data of each object after image processing is combined withthe image data of the background, so that image data in which a sense ofdepth or stereoscopic effect of an image is increased is obtained.

Further, one embodiment of the present invention utilizes humanperception of perspective: in the case where a plurality of objectsexists in a two-dimensional image, the outline of an object isreinforced compared to the outlines of the other objects, so that theobject seems to be located more on the front side than the otherobjects.

Specifically, in an image processing method according to one embodimentof the present invention, like the above-described image processingmethod, a relative distance between viewer's eye and any of the objectsor a front-back relation among the objects is determined. Then, theimage data of each object is processed so that the outline of an objectthat has a shorter relative distance from viewer's eye or an object thatis located more on the front side is reinforced. Then, similarly to theabove-described image processing method, the image data of each objectafter image processing is combined with the image data of thebackground, so that image data in which a sense of depth or stereoscopiceffect of an image is increased is obtained.

Further, one embodiment of the present invention may utilize humanperception of perspective: in the case where a plurality of objectsexists in a two-dimensional image, a vanishing point is assumed to be inthe two-dimensional image, and an object is located apart from thevanishing point, so that the object seems to be located more on thefront side than the other objects.

Specifically, an image processing method according to one embodiment ofthe present invention may obtain image data in which a sense of depth orstereoscopic effect of an image is increased by a method in which, likethe above described image processing method, the image data of eachobject is processed, an object in which a relative distance fromviewer's eye is shorter or an object that is located more on the frontside is moved to be located apart from the vanishing point, and theimage data of each object after image processing is combined with theimage data of the background.

An image processing method and a display device utilizing the imageprocessing method according to one embodiment of the present inventionare not like a method and a display device utilizing binocular parallaxfor example, in which a sense of depth or stereoscopic effect of animage is obtained by difference between an image data for a right eyeand an image data for a light eye. That is, with the above-describedimage processing method and the above-described display device utilizingthe image processing method, even in the case where one eye sees animage formed, a human can feel a sense of depth or stereoscopic effect.Accordingly, a method in one embodiment of the present invention, unlikea method utilizing the binocular parallax, does not necessary to displayan image for a right eye and an image for a left eye alternately orsimultaneously, therefore, a sense of depth or stereoscopic effect ofthe image can be increased while the number of writing of images to apixel portion during one frame period is reduced or the number of pixelsthat contribute to image display is not reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings:

FIG. 1 is a block diagram showing a configuration of a display device;

FIG. 2 shows an image processing procedure;

FIGS. 3A and 3B each show an example of an image;

FIGS. 4A to 4C each show an example of an image;

FIG. 5 shows an object A and feature points thereof;

FIGS. 6A and 6B each show an example of an image;

FIG. 7 is a circuit diagram of a pixel portion;

FIG. 8 is a top view of a pixel;

FIG. 9 is a cross-sectional view of a pixel;

FIG. 10 is a perspective view of a display device; and

FIGS. 11A to 11C each show an electronic device.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, embodiments of the present invention are described indetail with reference to the drawings. Note that the present inventionis not limited to the following description and it is easily understoodby those skilled in the art that the mode and details can be variouslychanged without departing from the scope and spirit of the presentinvention. Accordingly, the present invention should not be construed asbeing limited to the description of the embodiments below.

Embodiment 1

A display device according to one embodiment of the present invention isdescribed with reference to FIG. 1.

As shown in FIG. 1, a display device 100 according to one embodiment ofthe present invention includes an image-processing portion 101 and adisplay portion 102. The image-processing portion 101 processes an imagedata 103 input to the display device 100, and generates an image signal.The display portion 102 displays an image according to the image signal.

The image-processing portion 101 shown in FIG. 1 includes a control unit104, an arithmetic unit 105, an input portion 106, a buffer memory unit107, a nonvolatile memory unit 108, a display portion controller 109,and the like.

The control unit 104 is a circuit that controls all operations of thearithmetic unit 105, the input portion 106, the buffer memory unit 107,the nonvolatile memory unit 108, and the display portion controller 109that are included in the image-processing portion 101.

The arithmetic unit 105 is a logic circuit that performs arithmeticoperations such as logic operations, four arithmetic operations, and thelike.

The input portion 106 is an interface that converts a format for theimage data 103 that is input to the display device 100 into a format forthe display device 100.

The nonvolatile memory unit 108 includes a database 110. The database110 is a group of data in which a feature amount used for identificationof an object (object recognition) and data of the size of an object thatis a model are correlated with each other. Further, a variety of dataused for an arithmetic operation in the arithmetic unit 105, aninstruction that is carried out in the control unit 104, and the likeare stored in the nonvolatile memory unit 108.

The buffer memory unit 107 has a function of storing a variety of datatemporarily. Specifically, the buffer memory unit 107 may include a datacache, an instruction cache, a program counter, an instruction register,and a resister file. The data cache temporarily stores frequently useddata. The instruction cache temporarily stores a frequently usedinstruction of instructions (programs) sent to the control unit 104. Theprogram counter stores an address of an instruction to be carried outnext by the control unit 104. The instruction register stores aninstruction to be carried out next by the control unit 104. The resisterfile stores data read from the nonvolatile memory unit 108, dataobtained during an arithmetic operation in the arithmetic unit 105, dataobtained by an arithmetic operation in the arithmetic unit 105, or thelike.

The control unit 104 decodes an instruction input from the buffer memoryunit 107, and controls all operations of the arithmetic unit 105, theinput portion 106, the buffer memory unit 107, the nonvolatile memoryunit 108, and the display portion controller 109 according to theinstruction decoded, and processes the image data 103 that is input tothe image-processing portion 101.

The display portion controller 109 generates an image signal tospecifications of the display portion 102 with the use of the image data103 that is processed. The image signal generated is supplied to thedisplay portion 102. The display portion controller 109 has a functionof supplying a driving signal, such as a start pulse signal or a clocksignal, for controlling driving of the display portion 102 or a powersupply potential to the display portion 102.

The display portion 102 includes a pixel portion 111 that displays animage with the use of an image signal and a driver circuit 112 thatcontrols operation of the pixel portion 111. A device displaying animage by controlling a gray scale of each pixel (e.g., a liquid crystaldisplay device, a light-emitting device containing a light-emittingelement such as an organic light-emitting element (OLED), electronicpaper, a digital micromirror device (DMD), a plasma display panel (PDP),or a field emission display (FED)) can be used for the display portion102.

Next, an image processing method, which is performed in the displaydevice 100 shown in FIG. 1, according to one embodiment of the presentinvention is described.

FIG. 2 shows an example of an image processing procedure according toone embodiment of the present invention. First, as shown in FIG. 2, theimage data 103 is input to the image-processing portion 101 (A01: inputof image data). FIG. 3A shows an example of an image that is included inthe image data 103 before image processing.

The image data 103 may correspond to a full-color image or a monochromeimage. In the case where the image data 103 corresponds to a full-colorimage, a plurality of image data for each hue is included in the imagedata 103.

Note that in this specification, a full-color image refers to an imagedisplayed with gray scales of a plurality of colors having differenthues. In addition, a monochrome image refers to an image displayed witha gray scale of a color having a single hue.

In the image-processing portion 101, an object included in the imagedata 103 that is input to the image-processing portion 101 is extracted(A02: extraction of an object). An object can be extracted by extractingthe outline of an object. By the extraction of the object, the imagedata 103 is separated into image data of the object and image data of abackground. FIG. 3B shows an example in which an object A, an object B,and an object C are extracted from an image included in the image data103 shown in FIG. 3A.

Then, each of the feature amounts of the objects extracted is obtained(A03: obtainment of a feature amount of each object). A feature amountis obtained by converting features that are useful for objectrecognition of the object into values. A feature amount obtained from ashape, a color, a gray scale, texture, or the like can be used forobject recognition. A known method can be used for obtaining a featureamount. For example, in the case where a feature amount is obtained froma shape, feature points included in the outline of an object areextracted, and the positions of the feature points are used as a featureamount. Alternatively, feature points included in the outline of acomponent of an object are extracted, and the positions of the featurepoints are used as a feature amount.

Note that when an outline is assumed to include numerous outline points,an outline point where a slope of a tangent line markedly changes may beextracted as a feature point. Alternatively, in the case where outlinepoints that have equal slopes of tangent lines are continuously present,one of the outline points may be extracted as a feature point.

FIG. 5 shows an object A and a plurality of feature points 130 extractedfrom the outline of the object A. FIG. 5 illustrates the case where,among outline points forming the outline of the object A, only outlinepoints where a slope of a tangent line markedly changes are extracted asfeature points.

Note that one or more kinds of feature amounts may be used for objectrecognition of the object. Accuracy of object recognition can beincreased with the use of a plurality of kinds of feature amounts.

Then, each of the obtained feature amounts of the objects is comparedwith the database 110, so that the size of each object is obtained (A04:obtainment of the size of each object by comparing feature amounts withdatabase). For example, in the case where the object A is a human, dataof a human size can be obtained by comparing the feature amounts withthe database 110.

Note that even in the case where several objects are recognized as thesame object, the sizes of the objects have individual variability insome cases. In the case where the sizes of the objects have individualvariability, data with a wide range of sizes or data with an averagedsize may be prepared in the database 110.

Even in the case where an object does not exist (e.g., product of humanimagination), it is possible that the sizes of the objects are set by adesigner as appropriate, and the data of the size is prepared in thedatabase 110.

The data of the size of an object stored in the database 110 may be dataof the length, the width, the area, or the like of the object or acomponent included in the object.

The data of the size of each of the plurality of objects is obtained,and is compared with the size of each object in the image data 103 fordetermining a front-back relation among the objects (A05: determinationof a front-back relation among objects). For example, when the ratio ofa size D₁ of an object in the image data 103 to a size D₀ of an objectobtained from the database 110 is higher than the ratio of D₁ to D₀ ofthe other objects, the object is determined to be located more on thefront side than the other objects.

In the case of the image data shown in FIG. 3B, the object A has thehighest ratio of D₁ to D₀ and the object C has the lowest ratio of D₁ toD₀. Accordingly, the object A is located on the front side, the object Bis located more on the back side than the object A, and the object C islocated more on the back side than the object B.

Note that when a front-back relation among the objects is determined,the image data 103 may be assumed to have perspective, and the ratio ofa relative distance between viewer's eye and any of the objects may becalculated. The position of viewer's eye can be determined by a designeras appropriate.

Then, image data of each of the objects in the image data 103 isprocessed, so that the outlines of the objects that are located on thefront side are exaggerated and the objects that are located on the frontside are enlarged (A06: image processing of image data of objects). Themore an object is located on the front side, the more the outline of theobject is reinforced. Further, the more an object in the image data 103is located on the front side, the more the object is enlarged.

FIG. 4A illustrates an object A before image processing and an object Aafter image processing. As shown in FIG. 4A, the outline of the object Ais reinforced and the object A is enlarged by the image processing.Further, FIG. 4B illustrates an object B before image processing and anobject B after image processing. As shown in FIG. 4B, the outline of theobject B is reinforced and the object B is enlarged by the imageprocessing. Note that the outline of the object A that is located moreon the front side than the object B is more reinforced than the outlineof the object B, and the magnification rate of the object A is higherthan that of the object B.

The degree of the exaggeration of the outline and the magnification ratecan be determined in advance according to the order of the objects inthe front-back relation. The degree of the exaggeration of the outlineand the magnification rate of the object are considered in the casewhere m objects exist in image data (m is a natural number of 2 or more)and the positions of the objects are different from each other in termsof the front-back relation. For example, the outline of the object thatis in the m th position from the front side, which is an object locatedon the back side, is not reinforced, or the object is not enlarged. Thedegree of the exaggeration of the outline or the magnification rate ofthe object that is in the (m−1) th position from the front side is ttimes that of the object in the am th position (t is 1 or more). Thedegree of the exaggeration of the outline or the magnification rate ofthe object which is in the (m−2) th position from the front side is t²times that of the m th object. The degree of the exaggeration of theoutline or the magnification rate of the object which is in the (m−3) thposition from the front side is t³ times that of the m th object. Inthis manner, in accordance with the order of the objects, the more anobject is located on the front side, the higher the degree of theexaggeration of the outline or the magnification rate of the object is.In the case of such a structure, the degree of the exaggeration of theoutline of the object which is n th from the front side (n is a naturalnumber of 2 or more and m or less) or the magnification rate thereof canbe t^((m−n)).

Alternatively, the degree of the exaggeration of the outline and themagnification rate can be determined by a relative distance betweenviewer's eye and any of the objects.

Then, the image data of each object after image processing is combinedwith the image data of the background (A07: image processing ofcombining image data of each object with image data of a background). Inthis embodiment, image processing for obtaining stereoscopic vision(e.g., exaggeration of an outline or enlargement of a size) is notperformed on the object that is located on the back side among the otherobjects; however, the image processing may be performed on the imagedata of the object that is located on the back side.

FIG. 4C shows an image included in image data obtained by combining thedata of the object C and the background on which image processing is notperformed and the data of the objects A and B shown in FIGS. 4A and 4B,respectively, on which image processing is performed.

By the image processing method, image data with an increased sense ofdepth or increased stereoscopic effect can be obtained.

Note that in the above-described steps A06 (image processing of imagedata of objects) and A07 (image processing of combining image data ofobjects with image data of a background), a known interpolating processsuch as linear interpolation or the nearest-neighbor interpolation maybe used.

Further, in the step A06 (image processing of image data of objectslocated on a front side), an image before image processing is similar toan image after image processing. In an embodiment of the presentinvention, in the step A07 (image processing of combining image data ofobjects with image data of a background), the data of the object afterimage processing may be combined with the image data of the backgroundso that the center of the similarity is inside the object before imageprocessing; or the center of the similarity may be at the otherpositions.

In the latter case, for example, a vanishing point is assumed to be inthe image included in the image data 103 before image processing. Theobject after image processing is located so that the center point of anobject after image processing is located to be on a line connecting thecenter point of an object before image processing and the vanishingpoint and the distance between the center point of the object afterimage processing and the vanishing point is longer than the distancebetween the center point of the object before image processing and thevanishing point. Further, when an object is located more on the frontside, the distance between the center point of an object before imageprocessing and the center point of the object after image processing islengthened.

FIG. 6A illustrates the case where a vanishing point 131 is provided inan image included in the image data 103 before image processing.Further, a center point 132 corresponds to a center of gravity of theobject A before image processing, and a center point 133 corresponds toa center of gravity of the object B before image processing. A dashedand dotted line 134 connects the vanishing point 131 and the centerpoint 132, and a dashed and dotted line 135 connects the vanishing point131 and the center point 133.

FIG. 6B corresponds to an image included in image data obtained bycombining the data of the object A after image processing, the data ofthe object B after image processing, and the data of the background. Acenter point 136 corresponds to a center of gravity of the object Aafter image processing, and is located on the dashed and dotted line134. The distance between the vanishing point 131 and the center point136 is longer than the distance between the vanishing point 131 and thecenter point 132. A center point 137 corresponds to a center of gravityof the object B after image processing, and is located on the dashed anddotted line 135. The distance between the vanishing point 131 and thecenter point 137 is longer than the distance between the vanishing point131 and the center point 133.

With such a structure, when the center point of an object is locatedmore on the front side, the distance between the vanishing point and thecenter point of the object is longer; therefore, the object seems to belocated more on the front side than the other objects. Accordingly, asense of depth or stereoscopic effect of an image can be increased.

Note that FIGS. 6A and 6B illustrate the case where the center points ofgravity of the objects are assumed to be the center points thereof andpositions of the objects after image processing are determined; however,the positions of the center points of the objects are not limitedthereto as long as they are inside of the objects. Thus, a designer candetermine the positions of the center points as appropriate.

Embodiment 2

A light-emitting element such as an organic light-emitting element is adisplay element that emits light when supplied with current, and thushas a high contrast ratio. Accordingly, when a light-emitting devicewith the light-emitting element is used as a display portion of adisplay device according to one embodiment of the present invention, animage with a high sense of depth or high stereoscopic effect can bedisplayed.

In this embodiment, the structure of the pixel portion 111 is describedin the case where a light-emitting device is used as the display portion102 in the display device 100 according to one embodiment of the presentinvention shown in FIG. 1. FIG. 7 is an example of a specific circuitdiagram of the pixel portion 111.

Note that the names of the source terminal and the drain terminal of atransistor interchange depending on the polarity of the transistor or adifference between the levels of potentials applied to the electrodes.In general, in an n-channel transistor, an electrode to which a lowpotential is applied is called a source terminal, and an electrode towhich a high potential is applied is called a drain terminal. Further,in a p-channel transistor, an electrode to which a low potential isapplied is called a drain terminal, and an electrode to which a highpotential is applied is called a source terminal. Hereinafter, one of asource electrode and a drain electrode is a first terminal and the otheris a second terminal, and a structure of the pixel portion 111 isdescribed below.

In addition, “source terminal” of a transistor means a source regionthat is a part of an active layer or a source electrode connected to anactive layer. Similarly, “drain terminal” of a transistor means a drainregion that is a part of an active layer or a drain electrode connectedto an active layer.

Note that “connection” in this embodiment means electrical connectionand corresponds to the state in which current, voltage, or a potentialcan be supplied or transmitted. Accordingly, a connection state meansnot only a state of direct connection but also a state of indirectconnection through a circuit element such as a wiring, a resistor, adiode, or a transistor so that current, a potential, or voltage can besupplied or transmitted.

In addition, even when different components are connected to each otherin a circuit diagram, there is actually a case where one conductive filmhas functions of a plurality of components such as a case where part ofa wiring serves as an electrode. As used herein, “connection” in thisembodiment includes the case where one conductive film has functions ofa plurality of elements.

As shown in FIG. 7, the pixel portion 111 includes signal lines S1 toSx, scan lines G1 to Gy, and power supply lines V1 to Vx. A pixel 140has one of the signal lines S1 to Sx, one of the scan lines G1 to Gy,and one of the power supply lines V1 to Vx.

In each of the pixels 140, a transistor 141 has a gate electrodeconnected to the scan line Gj (j is one of 1 to y). The transistor 141has a first terminal connected to the signal line Si (i is one of 1 tox) supplied with an image signal, and a second terminal connected to thegate electrode of a transistor 142. The transistor 142 has a firstterminal connected to the power supply line Vi supplied with a powersupply potential, and a second terminal connected to the pixel electrodeof a light-emitting element 143.

A common potential COM is supplied to the common electrode of thelight-emitting element 143.

FIG. 7 illustrates the case where the pixel 140 includes a storagecapacitor 144. The storage capacitor 144 is connected to a gateelectrode of the transistor 142. The storage capacitor 144 retains thepotential of the gate electrode of the transistor 142. Specifically, oneof electrodes of the storage capacitor 144 is connected to the gateelectrode of the transistor 142 and the other is connected to a nodesupplied with a fixed potential, e.g., the power supply line Vi.

FIG. 7 illustrates the case where the transistor 141 and the transistor142 are n-channel transistors; however, each of the transistors may beeither an n-channel transistor or a p-channel transistor.

Further, the pixel 140 may further have another circuit element such asa transistor, a diode, a resistor, a storage capacitor, or an inductoras needed.

Note that the transistor 141 includes at least a gate electrode on oneside of an active layer. Alternatively, the transistor 141 may include apair of gate electrodes with the active layer interposed therebetween.In the case where the transistor 141 includes a pair of gate electrodeswith the active layer interposed therebetween, one of the gateelectrodes is connected to a signal line, and the other of the gateelectrodes (back gate electrode) may be in a floating state (i.e.,electrically isolated) or may be supplied with a potential from otherelements. In the latter case, potentials at the same level may beapplied to the pair of electrodes, or a fixed potential such as a groundpotential may be applied only to the back gate electrode. The level ofthe potential applied to the back gate electrode is controlled, so thatthe threshold voltage of the transistor 141 can be controlled.

In addition, the transistor 141 may be either a single-gate transistorthat includes a single gate electrode and a single channel formationregion, or a multi-gate transistor that includes a plurality of gateelectrodes electrically connected to each other and thus includes aplurality of channel formation regions.

Next, an operation of the pixel portion 111 shown in FIG. 7 isdescribed.

First, during the write period, the scan lines G1 to Gy are sequentiallyselected. For example, the scan line Gj is selected and the transistor141 that has a gate electrode connected to the scan line Gj is turnedon. Since the transistors 141 are turned on, the potentials of imagesignals that have been input to the signal lines S to Sx are supplied tothe gate electrodes of the corresponding transistors 142. Then, when theselection of the scan line Gj is terminated, the transistor 141 isturned off, and the potential of the image signal is held in the gateelectrode of the transistor 142.

Note that during the write period, the common potential COM is suppliedto the common electrodes of the light-emitting elements 143. Thelight-emitting state of the light-emitting element 143 is determined bythe potential of the image signal. Specifically, if the transistors 142are on in accordance with the potentials of the image signals, thelight-emitting elements 143 are supplied with current, and then emitlight. The amount of current supplied to the light-emitting elements 143greatly depends on the drain current of the transistor 142. Thus, theluminance of the light-emitting elements 143 is determined by thepotential of the image signal. In contrast, if the transistors 142 areoff in accordance with the potentials of the image signals, supply ofcurrent to the light-emitting elements 143 is not performed, so that thelight-emitting elements 143 do not emit light.

Next, the write period is finished and the holding period is started, sothat the transistor 141 is turned off. Then, the potential of the imagesignal that has been supplied to the gate electrode of the transistor142 during the write period is retained by the storage capacitor 144.Accordingly, the light-emitting element 143 maintains a light-emittingstate determined during the write period.

With the above-described operation, the pixel portion 111 can display animage.

Next, an example of a specific structure of the pixel 140 is described.FIG. 8 shows an example of a top view of the pixel 140. Note thatinsulating films are omitted in a top view shown in FIG. 8 in order toshow the layout of the pixel 140 clearly. Further, an electroluminescentlayer and a common electrode that are provided over a pixel electrodeare omitted in a top view shown in FIG. 8 in order to show the layout ofthe pixel 140 clearly.

In FIG. 8, a semiconductor film 801 and a semiconductor film 802 thatfunction as active layers of the transistor 141 and the transistor 142,respectively, are provided over a substrate having an insulatingsurface. The semiconductor film 801 also functions as one of theelectrodes of the storage capacitor 144.

A conductive film 803 that functions as a gate electrode of thetransistor 141 and a scan line is provided over the semiconductor film801 with a gate insulating film interposed therebetween. Further, aconductive film 804 that functions as one of the electrodes of thestorage capacitor 144 is provided over the semiconductor film 801 withthe gate insulating film interposed therebetween. A portion in which theconductive film 804 overlaps with the semiconductor film 801 with thegate insulating film interposed therebetween corresponds to the storagecapacitor 144. Still further, a conductive film 805 that functions asthe gate electrode of the transistor 142 is provided over thesemiconductor film 802 with the gate insulating film interposedtherebetween.

A first interlayer insulating film is formed over the conductive film803, the conductive film 804, and the conductive film 805. A conductivefilm 806 that functions as the signal line, a conductive film 807 thatfunctions as a power supply line, a conductive film 808, and aconductive film 809 are provided over the first interlayer insulatingfilm.

The conductive film 806 is connected to the semiconductor film 801 viaan opening portion 810 formed in the first interlayer insulating filmand the gate insulating film. The conductive film 807 is connected tothe semiconductor film 802 via an opening portion 811 formed in thefirst interlayer insulating film and the gate insulating film. Theconductive film 808 is connected to the semiconductor film 802 via anopening portion 812 formed in the first interlayer insulating film andthe gate insulating film, and is connected to the conductive film 805via an opening portion 813 formed in the first interlayer insulatingfilm. The conductive film 809 is connected to the semiconductor film 802via an opening portion 814 formed in the first interlayer insulatingfilm and the gate insulating film.

A second interlayer insulating film is formed over the conductive film806, the conductive film 807, the conductive film 808, and theconductive film 809. A conductive film 815 that functions as the pixelelectrode is provided over the second interlayer insulating film. Theconductive film 815 is connected to the conductive film 809 via anopening portion 816 formed in the second interlayer insulating film.

Note that in a display device according to one embodiment of the presentinvention, a color filter method in which a color filter and alight-emitting element that emits a single color such as white are usedin combination in order to display a full-color image can be employedfor the pixel portion 111 shown in FIG. 1. Alternatively, a method inwhich a full-color image is displayed with the use of a plurality oflight-emitting elements emitting light with different hues can beemployed. This method is called a separate coloring method because theelectroluminescent layer that is provided between a pair of electrodesincluded in the light-emitting element is separately colored by itscorresponding color.

In the case of the separate coloring method, generally, theelectroluminescent layer is separately colored by an evaporation methodwith the use of a mask such as a metal mask. Therefore, the size of thepixel depends on the separate coloring accuracy of theelectroluminescent layer. On the other hand, in the case of the colorfilter method, unlike the separate coloring method, theelectroluminescent layer does not need to be separately colored.Accordingly, in the case of the color filter method, as compared to theseparate coloring method, the pixel can be easily reduced in size, andthe pixel portion with high high-definition can be fabricated. The pixelportion with high high-definition has an advantage in that a sense ofdepth or stereoscopic effect of an image is increased. Accordingly, interms of increasing a sense of depth or stereoscopic effect, alight-emitting device formed with the color filter method is moresuitable than a light-emitting device with the separate coloring methodfor a display device according to one embodiment of the presentinvention.

A bottom emission type light-emitting device in which light of thelight-emitting element is extracted from the substrate (elementsubstrate) side or a top emission structure in which light of thelight-emitting element is extracted from a side opposite to the elementsubstrate side can be used for a light-emitting device. In the case of atop emission type light-emitting device, light emitted from thelight-emitting element is not blocked by a variety of elements such as awiring, a transistor, or a storage capacitor; therefore, thelight-extraction efficiency of light from a pixel of the light-emittingdevice can be higher than that of a bottom emission type light-emittingdevice. Accordingly, a top emission type light-emitting device can havehigh luminance even when the light-emitting element is supplied with lowcurrent value, and has an advantage in that the lifetime of thelight-emitting element is prolonged.

Further, a display device according to one embodiment of the presentinvention may have a microcavity (micro optical resonator) structure inwhich light emitted from an electroluminescent layer is resonated in alight-emitting element. With a microcavity structure, thelight-extraction efficiency of light with a particular wavelength fromthe light-emitting element can be increased, so that the luminance andthe color purity of the pixel portion can be improved.

FIG. 9 shows an example of a cross-sectional view of a pixel having amicrocavity structure. FIG. 9 shows part of the cross section of a pixelemitting red light, part of the cross section of a pixel emitting bluelight, and part of the cross section of a pixel emitting green light.Each of the three parts of cross sections of pixels corresponds to adashed and dotted line A1-A2 in the top view of the pixel 140 shown inFIG. 8.

Specifically, FIG. 9 shows a pixel 140 r emitting red light, a pixel 140g emitting green light, and a pixel 140 b emitting blue light. The pixel140 r, the pixel 140 g, and the pixel 140 b include a pixel electrode815 r, a pixel electrode 815 g, and a pixel electrode 815 b,respectively. The pixel electrode 815 r, the pixel electrode 815 g, andthe pixel electrode 815 b are connected to the transistor 142 in thepixel 140 r, the transistor 142 in the pixel 140 g, and the transistor142 in the pixel 140 b, respectively, via the conductive film 809 over asubstrate 840.

A partition wall 830 having an insulating film is provided over each ofthe pixel electrode 815 r, the pixel electrode 815 g, and the pixelelectrode 815 b. The partition wall 830 has an opening portion, and inthe opening portion, each of the pixel electrode 815 r, the pixelelectrode 815 g, and the pixel electrode 815 b is partly exposed. Inorder to cover the exposed portions, an electroluminescent layer 831 anda common electrode 832 that transmits visible light are stacked over thepartition wall 830 in this order.

A portion in which the pixel electrode 815 r, the electroluminescentlayer 831, and the common electrode 832 overlap with each othercorresponds to a light-emitting element 841 r emitting red light. Aportion in which the pixel electrode 815 g, the electroluminescent layer831, and the common electrode 832 overlap with each other corresponds toa light-emitting element 841 g emitting green light. A portion in whichthe pixel electrode 815 b, the electroluminescent layer 831, and thecommon electrode 832 overlap with each other corresponds to alight-emitting element 841 b emitting blue light.

A substrate 842 faces the substrate 840 with the light-emitting element841 r, the light-emitting element 841 g, and the light-emitting element841 b interposed therebetween. A coloring layer 843 r corresponding tothe pixel 140 r, a coloring layer 843 g corresponding to the pixel 140g, and a coloring layer 843 b corresponding to the pixel 140 b areformed over the substrate 842. The coloring layer 843 r is a layer thathas higher transmittance of light in a wavelength region correspondingto red than light in the other wavelength regions. The coloring layer843 g is a layer that has higher transmittance of light in a wavelengthregion corresponding to green than light in the other wavelengthregions. The coloring layer 843 b is a layer that has highertransmittance of light in a wavelength region corresponding to blue thanlight in the other wavelength regions.

Further, an overcoat 844 is provided over the substrate 842 to cover thecoloring layer 843 r, the coloring layer 843 g, and the coloring layer843 b. The overcoat 844 is a layer that transmits visible light, isprovided for protecting the coloring layer 843 r, the coloring layer 843g, and the coloring layer 843 b, and is preferably formed with a resinmaterial having high level of planarity. The combination of the overcoat844 and any one of the coloring layers 843 r, 843 g, and 843 b may beregarded as a color filter. Further, each of the coloring layer 843 r,the coloring layer 843 g, and the coloring layer 843 b may also beregarded as a color filter.

As shown in FIG. 9, in one embodiment of the present invention, aconductive film 845 r with high visible-light reflectance and aconductive film 846 r with visible-light transmittance higher than thatof the conductive film 845 r are stacked in this order and used as thepixel electrode 815 r. A conductive film 845 g with high visible-lightreflectance and a conductive film 846 g with visible-light transmittancehigher than that of the conductive film 845 g are stacked in this orderand used as the pixel electrode 815 g. The thickness of the conductivefilm 846 g is smaller than that of the conductive film 846 r. Further, aconductive film 845 b with high visible-light reflectance is used as thepixel electrode 815 b.

Accordingly, in a light-emitting device shown in FIG. 9, in thelight-emitting element 841 r, an optical path length of light emittedfrom the electroluminescent layer 831 is determined by the distancebetween the conductive film 845 r and the common electrode 832. Further,in the light-emitting element 841 g, an optical path length of lightemitted from the electroluminescent layer 831 is determined by thedistance between the conductive film 845 g and the common electrode 832.Still further, in the light-emitting element 841 b, an optical pathlength of light emitted form the electroluminescent layer 831 isdetermined by the distance between the conductive film 845 b and thecommon electrode 832.

One embodiment of the present invention employs a microcavity structurein which the above optical path length is adjusted in accordance withwavelength of light each corresponding to the light-emitting element 841r, the light-emitting element 841 g, and the light-emitting element 841b, so that light emitted from the electroluminescent layer 831 isresonated in the light-emitting element. For example, in FIG. 9, when adistance L is the distance between the common electrode 832 and any oneof the conductive films 845 r, 845 g, and 845 b, a refractive index n isthe refractive index of the electroluminescent layer 831, and awavelength λ is the wavelength of light that is expected to resonate,the product of the distance L and the index n is preferably (2N−1)/4times the wavelength λ (N is a natural number).

With the employment of the microcavity structure for one embodiment ofthe present invention, the intensity of light having a wavelengthcorresponding to red light in light emitted from the light-emittingelement 841 r is increased by resonance. Accordingly, the color purityand the luminance of red light obtained through the coloring layer 843 rare increased. Further, with the employment of the microcavity structurefor one embodiment of the present invention, the intensity of lighthaving a wavelength corresponding to green light in light emitted fromthe light-emitting element 841 g is increased by resonance. Accordingly,the color purity and the luminance of green light obtained through thecoloring layer 843 g are increased. Still further, with the employmentof the microcavity structure for one embodiment of the presentinvention, the intensity of light having a wavelength corresponding toblue light in light emitted from the light-emitting element 841 b isincreased by resonance. Accordingly, the color purity and the luminanceof blue light obtained through the coloring layer 843 b are increased.

FIG. 9 shows a structure in which pixels emitting three colors of red,green, and blue are used; however, one embodiment of the presentinvention is not limited to the structure. A combination of the colors,which is used in one embodiment of the present invention, may includefour colors of red, green, blue, and yellow, or three colors of cyan,magenta, and yellow. Alternatively, the combination of the colors mayinclude six colors of pale red, pale green, pale blue, deep red, deepgreen, and deep blue. Alternatively, the combination of the colors maybe six colors of red, green, blue, cyan, magenta, and yellow.

Note that, for example, colors that can be exhibited using the pixels ofred, green, and blue are limited to colors existing in the triangle madeby the three points on the chromaticity diagram, which correspond to theemission colors of the respective light sources. Therefore, as in thecase where the pixels of red, green, blue, and yellow are used, byadditionally providing a light source of a color existing outside thetriangle on the chromaticity diagram, the range of the colors which canbe expressed in the light-emitting device can be expanded, so that colorreproducibility can be enhanced.

In FIG. 9, in the light-emitting element 841 b whose wavelength λ oflight is the shortest among the light-emitting elements 841 r, 841 g,and 841 b, the conductive film 845 b with high visible-light reflectanceis use as a pixel electrode. Further, the conductive film 846 r and theconductive film 846 g that have thicknesses different from each otherare used in the light-emitting element 841 r and the light-emittingelement 841 g, respectively, so that the optical path length isadjusted. In one embodiment of the present invention, a conductive filmwith high visible-light transmittance, such as the conductive film 846 ror the conductive film 846 g may also be provided over the conductivefilm 845 b with high visible-light reflectance in the light-emittingelement 841 b whose wavelength λ is the shortest. However, themanufacturing process of the case where the conductive film 845 b withhigh visible-light reflectance is use as a pixel electrode in thelight-emitting element 841 b whose wavelength λ is the shortest as shownin FIG. 9 is easier and more preferable than the manufacturing processof the case where the conductive film with high visible-lighttransmittance is used as the pixel electrode in all of thelight-emitting elements.

Note that the conductive film 845 b with high visible-light reflectanceoften has smaller work function than the conductive film 846 r and theconductive film 846 g that have high visible-light transmittance.Therefore, in the light-emitting element 841 b whose wavelength λ is theshortest, compared to the light-emitting element 841 r and thelight-emitting element 841 g, hole injection from the pixel electrode815 b which is an anode to the electroluminescent layer 831 is noteasily performed, and the emission efficiency of the light-emittingelement 841 b tends to be low. Accordingly, in one embodiment of thepresent invention, a composite material that contains a substance havinga high hole-transport property and a substance having an acceptor(electron-accepting) property with respect to the substance having ahigh hole-transport property is preferably used as a layer of theelectroluminescent layer 831 which is in contact with the conductivefilm 845 b with high visible-light reflectance in the light-emittingelement 841 b whose wavelength λ is the shortest. The composite materialis formed to be in contact with the pixel electrode 815 b that is ananode, so that the hole injection from the pixel electrode 815 b to theelectroluminescent layer 831 is easily performed, and the emissionefficiency of the light-emitting element 841 b can be increased.

As the substance having an acceptor property,7,7,8,8-tetracyano-2,3,5,6-tetrafluoroquinodimethane (abbreviation:F₄-TCNQ), chloranil, and the like can be given. Further, a transitionmetal oxide can be given. In addition, oxides of metals belonging toGroups 4 to 8 in the periodic table can be also given. Specifically,vanadium oxide, niobium oxide, tantalum oxide, chromium oxide,molybdenum oxide, tungsten oxide, manganese oxide, rhenium oxide, or thelike is a preferable material because the acceptor property is high.Among these, molybdenum oxide is especially preferable because it isstable in the air, has a low hygroscopic property, and is easilytreated.

As the substance having a high hole-transport property used for thecomposite material, any of a variety of compounds such as an aromaticamine compound, a carbazole derivative, aromatic hydrocarbon, or a highmolecular compound (e.g., an oligomer, a dendrimer, or a polymer) can beused. The organic compound used for the composite material is preferablyan organic compound having a high hole-transport property. Specifically,a substance having a hole mobility of 10⁻⁶ cm²/Vs or higher ispreferably used. However, substances other than the above describedmaterials may also be used as long as the substances have higherhole-transport properties than electron-transport properties.

Each of the conductive film 845 r, the conductive film 845 g, and theconductive film 845 b which have high visible-light reflectance can beformed using a single layer or a stacked layer using aluminum, silver,an alloy containing such a metal material, or the like. Alternatively,each of the conductive film 845 r, the conductive film 845 g, and theconductive film 845 b can be formed by stacking a conductive film withhigh visible-light reflectance and a conductive film with a smallthickness (preferably 20 nm or less, more preferably 10 nm or less). Forexample, a thin titanium film or a thin molybdenum film is stacked overthe conductive film with high visible-light reflectance to form theconductive film 845 b, so that an oxide film can be prevented from beingformed over the surface of the conductive film with high visible-lightreflectance (e.g., an aluminum film, an alloy film containing aluminum,or a silver film).

As the conductive films 846 r and 846 g that have high visible-lighttransmittance, for example, indium oxide, tin oxide, zinc oxide, indiumtin oxide, or indium zinc oxide can be used.

The common electrode 832 can be formed by stacking a conductive filmwith a thickness thin enough to transmit light (preferably 20 nm orless, more preferably 10 nm or less) and a conductive film containing aconductive metal oxide. The conductive film with a thickness thin enoughto transmit light can be formed using a single layer or a stacked layerusing silver, magnesium, an alloy containing such a metal material, orthe like. As the conductive metal oxide, it is possible to use indiumoxide, tin oxide, zinc oxide, indium oxide-tin oxide, indium oxide-zincoxide, or any of these metal oxide materials containing silicon oxide.

Next, FIG. 10 shows an example of a perspective view of a display deviceaccording to one embodiment of the present invention. FIG. 10illustrates a display device in which a light-emitting device is used ina display portion.

The display device shown in FIG. 10 includes a display portion 1601, acircuit substrate 1602, and a connecting portion 1603.

The circuit substrate 1602 is provided with an image-processing portion,and a variety of signals or a power supply potential is input to thedisplay portion 1601 via the connecting portion 1603. As the connectingportion 1603, a flexible printed circuit (FPC) or the like can be used.In the case where a COF tape is used as the connecting portion 1603,part of the circuit of the image-processing portion or part of thedriver circuit included in the display portion 1601 is formed on a chipseparately prepared, and the chip may be connected to a COF tape by aCOF (chip on film) method.

This embodiment can be implemented in combination with the aboveembodiments.

Example

A display device according to one embodiment of the present inventioncan display an image with a high sense of depth or high stereoscopiceffect. Specifically, the display device according to one embodiment ofthe present invention can be applied to image display devices, laptopcomputers, or image reproducing devices provided with recording media(typically devices that reproduce the content of recording media such asdigital versatile discs (DVDs) and have displays for displaying thereproduced images). In addition to the above examples, as an electronicdevice including the display device according to one embodiment of thepresent invention, mobile phones, portable game machines, portableinformation terminals, e-book readers, video cameras, digital stillcameras, goggle-type displays (head mounted displays), navigationsystems, audio reproducing devices (e.g., car audio components anddigital audio players), copiers, facsimiles, printers, multifunctionprinters, automated teller machines (ATM), vending machines, and thelike can be given. Specific examples of such electronic appliances areshown in FIGS. 11A to 11C.

FIG. 11A illustrates a portable game machine including a housing 5001, ahousing 5002, an image display portion 5003, an image display portion5004, a microphone 5005, speakers 5006, operation keys 5007, a stylus5008, and the like. The display device according to one embodiment ofthe present invention can be used as the image display portion 5003 orthe image display portion 5004. The portable game machine is capable ofdisplaying an image with a high sense of depth or high stereoscopiceffect when a display device according to one embodiment of the presentinvention is used as the image display portion 5003 or 5004. Note thatalthough the portable game machine in FIG. 11A has the two image displayportions 5003 and 5004, the number of image display portions included inthe portable game machine is not limited to two.

FIG. 11B illustrates a laptop, which includes a housing 5201, an imagedisplay portion 5202, a keyboard 5203, a pointing device 5204, and thelike. The display device according to one embodiment of the presentinvention can be used for the image display portion 5202. The laptoppersonal computer is capable of displaying an image with a high sense ofdepth or high stereoscopic effect when a display device according to oneembodiment of the present invention is used as the image display portion5202.

FIG. 11C illustrates a portable information terminal, which includes ahousing 5401, an image display portion 5402, operation keys 5403, andthe like. The display device according to one embodiment of the presentinvention can be used as the image display portion 5402. The portableinformation terminal is capable of displaying an image with a high senseof depth or high stereoscopic effect when a display device according toone embodiment of the present invention is used as the image displayportion 5402.

As described above, the present invention can be widely applied to andused in electronic devices in a wide variety of fields.

This embodiment can be implemented in combination with embodiments abovementioned.

This application is based on Japanese Patent Application serial No.2011-067069 filed with Japan Patent Office on Mar. 25, 2011, the entirecontents of which are hereby incorporated by reference.

1. (canceled)
 2. An image processing method comprising the steps of:separating first image data of an image into image data of an object andimage data of background; obtaining data of a size of the object;processing the image data of the object to obtain an enlarged image dataof the object; and combining the enlarged image data of the object withthe image data of the background to generate second image data.
 3. Theimage processing method according to claim 2, further comprising thestep of obtaining a feature amount with data stored in a database. 4.The image processing method according to claim 2, wherein the processingstep and the combination step are carried on so that a sense of depth orstereoscopic effect of an image formed from the second image data isincreased compared to a sense of depth or stereoscopic effect of animage formed from the first image data.
 5. An image processing methodcomprising the steps of: separating first image data of an image intoimage data of an object and image data of background; obtaining data ofthe object; processing the image data of the object to change a degreeof an exaggeration of an outline of the object; and combining the imagedata of the object after the processing with the image data of thebackground to generate second image data.
 6. The image processing methodaccording to claim 5, further comprising the step of obtaining a featureamount with data stored in a database.
 7. The image processing methodaccording to claim 5, wherein the processing step and the combinationstep are carried on so that a sense of depth or stereoscopic effect ofan image formed from the second image data is increased compared to asense of depth or stereoscopic effect of an image formed from the firstimage data.
 8. An image processing method comprising the steps of:extracting image data of a first object and a second object from firstimage data; obtaining data of the first object and the second object;processing the image data of the first object and the second object tochange a degree of an exaggeration of an outline of one of the firstobject and the second object; and combining the image data of the firstobject and the second object after the processing with the image data ofa background of the first image data to generate second image data. 9.The image processing method according to claim 8, further comprising thestep of obtaining a feature amount with data stored in a database. 10.The image processing method according to claim 8, further comprising thestep of determining a front-back relation between the first object andthe second object.
 11. The image processing method according to claim 8,wherein the processing step and the combination step are carried on sothat a sense of depth or stereoscopic effect of an image formed from thesecond image data is increased compared to a sense of depth orstereoscopic effect of an image formed from the first image data. 12.The image processing method according to claim 8, wherein the one of thefirst object and the second object is enlarged compared to the other ofthe first object and the second object.
 13. The image processing methodaccording to claim 8, wherein the outline of the one of the first objectand the second object which is determined to be more on the front sideof the image are reinforced compared to the outline of the other of thefirst object and the second object which is determined to be more on aback side of the image.
 14. The image processing method according toclaim 8, wherein a distance between a vanishing point and a center pointof the one of the first object and the second object which is determinedto be more on the front side of the image is lengthened compared to adistance between the vanishing point and a center point of the other ofthe first object and the second object which is determined to be more ona back side of the image.